Pharmaceutical verification camera system and method

ABSTRACT

A pharmaceutical verification (PV) camera system that captures an image of the contents of a vial on an automated dispensing line is closed. Faster image processing time is achieved by utilizing a learning algorithm that stores camera parameters for a successful image associated with data for a prescription processed on the automated dispensing line. During processing of a prescription order, when the vial contents and availability of stored parameters is confirmed, the stored parameters are transmitted to the camera and an image of the vial contents is captured and stored. When a previously un-encountered or un-trained vial is detected, the camera engages the autofocus feature to capture an image. The learning algorithm evaluates the image based on feedback from one or more metric. Upon agreement with the metric standards, an image is accepted and archived and the camera parameters associated with that vial prescription are stored for later use.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/960,709 filed on Aug. 6, 2013, which is a non-provisional applicationof U.S. Provisional Patent Application Ser. No. 61/720,066, filed Oct.30, 2012. All aforementioned applications are hereby incorporated byreference in their entirety as if fully cited herein.

BACKGROUND

Many health benefit plan providers and retail pharmacies now offer theirclients the option of obtaining prescription drugs by mail. Mail orderpharmacies ship prescription drugs to a client's home so the client isnot required to visit a pharmacy and to fill a prescription in person.For clients with chronic conditions or other conditions that requiremaintenance drugs, a mail order prescription program is an attractivebenefit because it is more convenient for the clients and typically lessexpensive than obtaining prescription drugs at a neighborhood pharmacy.For many drugs, clients have the option of purchasing a drug fill in a60-day or even a 90-day supply at a lower cost than a 30-day supply.

Many mail order pharmacies use automated systems and dispensing lines toprocess and ship a high volume of prescriptions on a daily basis.Depending upon how the technology is implemented and deployed within amail order pharmacy, a substantial number of steps in the fulfillmentprocess may be automated and the need for human intervention minimized.Mail order pharmacies operated in the US, like their neighborhoodcounterparts, must be licensed in a state and are subject to numerousrules and regulations established by the licensing state's board ofpharmacy. One common requirement is that a pharmacy, whether aneighborhood pharmacy or a mail order pharmacy, must meet pharmacistverification for certain prescriptions. For automated mail orderpharmacies, pharmacist verification is a manual step that must beintegrated into the automated fulfillment process.

In many automated pharmacy systems, pharmaceutical verification isperformed by capturing and displaying at a workstation the verificationdata that the pharmacist needs to review and verify a prescriptionorder. The verification data typically includes prescribed drug datafrom the order (e.g., drug name, strength, dosage form, and quantityprescribed) and a digital image of a drug that has been dispensed into avial for shipment to the patient. The digital images are typicallyacquired from one or more digital camera systems that are integratedinto an automated dispensing line. The pharmacist reviews theprescription order data and image of the vial contents to confirm theproper drug has been added to the vial to be dispensed to the patient.

Although state boards of pharmacy typically do not require pharmacistverification for every prescription filled by a mail order pharmacy, theautomated system must capture a digital image of every prescription thatis filled so a record of the order and vial contents can be retrieved inthe event questions about processing of the order arise. When pharmacistverification is required, the digital image of the vial contents alongwith the prescription order data allows the pharmacist to confirm theproper drug has been dispensed. Therefore, it is important for the mailorder pharmacy to incorporate an image capture process into theautomated prescription dispensing line.

A pharmaceutical verification camera system is an important component ofan automated prescription dispensing system but frequently such camerasystems are also a bottleneck in the dispensing system. The capsules andpills that are dispensed vary in size, color, and shape, and therefore,require different camera settings to capture a clear image. Furthermore,the appearance of the capsules and pills within a vial can vary based onthe volume of the drug added to the vial. To capture a clear image ofevery filled vial, automated dispensing systems typically rely on thecamera's autofocus and automated color balancing features to determinethe appropriate settings (e.g., focus, white balance, and exposure time)for the image capture.

The camera's automated features are part of an open-loop system thatrelies completely on internal routines to converge on and deliver animage. The time required to change the auto-settings as well as capturemultiple images can take several seconds. A camera host computertypically waits for a period of time while the camera's embeddedcontroller optimizes the image. At the end of this arbitrary period, thesoftware assumes it has captured a valid image. The pass/fail result inthis context is only based on the existence of the image. There are noquality checks in place to confirm an image of sufficient quality hasbeen captured. The camera host saves the image to the archive andupdates the control system so the vial may progress through thepharmacy, but the quality of the image is unknown.

In many systems, the vials are queued for image processing simplybecause the camera cannot keep up with the volume of vials that areprocessed by the system. Therefore, there is a need for an improvedpharmaceutical verification camera system that reduces or eliminatesbottlenecks in an automated dispensing system.

SUMMARY

The present disclosure describes an improved pharmaceutical verification(PV) camera system that captures an image of the contents of a vial onan automated dispensing line. Faster image processing time is achievedby utilizing a learning algorithm that stores camera parameters for asuccessful image associated with data for a prescription order processedon the automated dispensing line. The stored camera parameters areapplied to later vials with the same configuration (e.g., drug and pillcount). The camera system further employs a series of checks andbalances to evaluate image quality. These checks and balances areimportant for both training the camera and also in maintaining imagequality after the training phase.

In an example embodiment, the camera applies stored parameters when itencounters a vial with contents that have been subjected to the learningalgorithm. When the vial contents and availability of stored parametersis confirmed, the stored parameters are transmitted to the camera and animage of the vial contents is captured and stored. When a previouslyun-encountered or un-trained vial is detected, the camera engages theautofocus feature of the camera. The autofocus function engages and thelearning algorithm evaluates the image based on feedback from one ormore metric. Upon agreement with the metric standards, an image isaccepted and archived and the camera parameters associated with thatvial prescription are stored for later use. When a similar vialprescription is encountered by the camera, the stored camera parametersare retrieved and relayed to the camera. The camera adjusts to thesesettings and captures a successful image without engaging the auto focusand other auto-setting features, thus saving image processing time andreducing the likelihood of bottlenecks.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a camera system component diagram according to an exampleembodiment;

FIG. 2 is first in-line camera system assembly drawing according to anexample embodiment;

FIG. 3 is a second in-line camera system assembly drawing according toan example embodiment;

FIG. 4 is a software state diagram according to an example embodiment;

FIG. 5 is a training image evaluation state diagram according to anexample embodiment; and

FIG. 6 is an image evaluation state diagram according to an exampleembodiment.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENT(S)

The disclosed pharmaceutical verification (PV) camera system and methodcaptures images of vial contents on an automated dispensing line. Thecamera system is conveyor mounted and comprises an expert machinelearning system and algorithm that stores camera parameters for asuccessful image associated with data for the prescription. The storedcamera parameters are applied to later vials with the same prescription.The captured images are archived and used for reference duringverification in addition to providing a record of the vial contents.

In an example embodiment, the algorithm:

-   -   Interprets unique prescription information about an incoming        order;    -   Determines if the prescription has been previously imaged;    -   Activates the camera image capture function;    -   Continuously evaluates the image quality prior to photo record        creation; and    -   Stores the associated parameters with the captured image for        forthcoming vials with similar characteristics.

In an example embodiment, software that comprises a camera learningalgorithm and other custom imaging processing functions is used tocontrol a digital camera. In an example embodiment, the control softwareis implemented as a library of functions. One of skill in the art wouldunderstand that in addition to employing a library, the features andfunctionality of the disclosed camera control software could beimplemented and/or integrated in a digital camera in a variety of ways,including incorporating the features and functionality into the camera'sstandard control software or providing it as an add-on.

In an example embodiment, the camera controller library providesfunctionality for a camera host interface that controls aconveyor-mounted camera used for vial imaging. The camera controllerlibrary comprises an employment capture process for processing of vialswith “known” contents and a training capture process for processing ofvials with “unknown” contents. In an example embodiment, the cameracontroller library uses prescription order information, in particularthe National Drug Code-NDC and pill count, for a pre-imaging processthat determines whether camera settings for the NDC/pill countcombination have been set.

In an example embodiment, a camera host interface initiates imaging of avial. When the vial enters a section of the automated dispensing linecontrolled by the camera host interface, it captures a puck containingthe vial, images the vial contents, and releases the puck. The camerahost interface reads an RFID tag for the vial and determines theprescription order data associated with the vial (e.g., NCD and pillcount). The NDC and pill count information is passed to the cameracontroller library. If the parameters are known, the employment captureprocess is initiated. If the parameters are unknown, the trainingcapture process is initiated.

In the training capture process, a plurality of images are captured andevaluated for conformance within specified tolerances. In an exampleembodiment, the training capture process captures and reviews thesharpness of multiple images. If the sharpness between images is withina specified tolerance, the image parameters are accepted and stored forsubsequent use. If the image sharpness varies substantially between theimages, the image parameters are not accepted and an exception handlingprocess is invoked. One of skill in the art would understand that imagequality may be measured using many well-known techniques. Improvementsin image quality may be achieved by minimizing optical aberrations suchas motion blur, focus blur, color clarity, and image sharpness anddepth. In an example embodiment, image quality is measured in terms of“blur” such that a blur detection scheme based on wavelet transform maybe used in the training capture process.¹

Referring to FIG. 1, a camera system component diagram according to anexample embodiment is shown. The camera system comprises a plurality ofhardware and software modules that interact to provide the disclosedfeatures and functionality. The hardware controls comprise a containeridentification reader, facility hardware interface, and automationsensors and actuators. An image processing module comprises a camerahost interface and camera controller library. The camera hardwarecomprises a streaming video component and onboard controller.

TABLE 1 Camera Components Module Responsibility Container IdentificationReading unique identification Reader (External to Camera) tag on vialcarrier Facility Hardware Interface Providing software with (External toCamera) the state of the sensors Providing software access to theactuators Low-level processing Automation Sensors & Monitor andmanipulate the Actuators (External to Camera) physical state of thepharmacy Camera Interface Host Interface with the pharmacy facility(alt. Camera Host) Interface with the order information Initiate imagecapture and finalize image capture Acquire image from camera OrderInformation Database of all information for current orders StreamingVideo Access to camera imaging sensor On-board Controller Access tocamera sensor and lens configuration Camera Controller Library Interfaceto hardware and controller for image capture process

The camera controller library provides unique functionality thatsubstantially improves image capture quality and processing times. Thecamera host supplies the camera controller library with orderinformation and a start signal. Control is then passed to the libraryuntil it completes processing and returns control to the camera host.

Referring to FIG. 2, a first in-line camera assembly drawing accordingto an example embodiment is shown. The camera system assembly 160comprises a support structure 172 and a camera mounting structure 162 towhich the camera 164 is attached. A light 166 is engaged during theimage capture phase to increase the quality of the images that arecaptured. The camera system assembly 160 is installed on an automateddispensing line and line control software interfaces with the camerahost software to capture and store images of vials 168 that aretransported on a conveyor of the automated dispensing line.

Referring to FIG. 3, a second in-line camera assembly drawing accordingto an example embodiment is shown. As vials travel on the conveyor 170,they are singulated 188 into an imaging position for image capture. Inan example embodiment, a singulating photoeye 186 is used in thesingulation process. Once a vial is singulated, it continues movingtoward the camera system assembly. The RFID for the vial is read by thecamera host interface using an RFID reader 190 integrated into theconveyor 170. A photoeye 182 and holdback 184 facilitate properalignment of the vial for imaging and the camera controller libraryinitiates a pre-imaging process to determine the vial contents based onthe RFID. The RFID is used to retrieve prescription order data for thevial.

One of skill in the art would understand that other variables such asvial size, vial color, etc. that impact the possible vialconfigurations, and therefore, the sharpness and quality of teach image,could be considered by the learning algorithm in determining cameraparameters. The NDC and pill count values are transmitted to the cameracontroller library that then determines whether stored camera parametersfor the NDC/pill count combination are available. If parameters for theNDC/pill count combination are available, they are communicated to thecamera API 162 to capture an image of the vial 180 positioned under thecamera. The image is captured and returned to the camera controllerlibrary which places the image in a repository for storage with theprescription order data. If parameters for the NDC/pill countcombination are not available, the camera controller library invokes thetraining capture process.

Referring to FIG. 4, a software state diagram according to an exampleembodiment is shown. When the camera system receives a vial, it readsthe contents via the order information stored in a shared memory 200.The software categorizes the information into a binary value: known orunknown 202. An unknown configuration is a combination with which thesoftware is unfamiliar. In this event, a training capture process 204 isspawned which is a slower, more rigorous capture process that acquires avalid image and parameters necessary to replicate that image. When thecamera encounters a known configuration or can infer one using aheuristic method, it spawns an employment capture process 206. Theemployment capture process is much simpler and faster because thecamera's parameters are known. It configures the camera and thencaptures the image. In an example embodiment, this frequently takesabout 1/10th the time of the training capture.

Vial Capture Learning Algorithm

Camera parameters are functions of the pill type, quantity, and vialsize. The learning algorithm borrows on concepts of artificial neuralnetworks and fuzzy logic and discovers the functions through training.The implementation constantly evaluates and optimizes itself duringcaptures to ensure the most optimal camera parameters are used. On asimple level, the training capture is a data update and the employmentcapture is a data look-up.

Training Evaluation

Referring to FIG. 5, a training image evaluation state diagram accordingto an example embodiment is shown. To train the learning algorithmeffectively, a high-quality image is used. The training process requiresan image of sufficient quality because the machine learning system islive and unsupervised. The pharmaceutical aspect compounds thedifficulties in training so the image filter must adapt to theconstantly changing conditions that manifest themselves as pill size,shape, and color. Additionally, it must perceive image quality whileavoiding false positives and negatives.

In an example embodiment, the training capture evaluation is a passivehigh-fidelity filter that acquires the best possible image from a livecamera stream by evaluating the presented image only. The camera'sinternal routines are employed to find the best possible image. Thisapproach results in a two-stage process:

-   1. Initialize the camera auto focus and color balancing and wait for    it to converge on an image; and-   2. Evaluate image for fitness and provide the control system with a    go/no go signal.

Initial 210: The camera is initialized for a capture. This capturemethod is utilized when the camera's memory has no prior information ona vial configuration. The camera is configured to maximize thelikelihood of convergence by setting the white balance and sensor gainsto a neutral state. The focus depth is set to a random level within thevial. The random depth ensures that the numerical method employed by theembedded controller never starts with the same initial conditions, whichmakes multiple solution computational divergences unlikely. The finalstep in initialization starts the internal routines in the camera'sembedded controller to auto focus and white balance.

Edge Debounce 212: In an example embodiment of the invention in whichthe camera controller is an open-controller, when the autofocus and/orcolor balancing routine are engaged, there is no direct feedback onfinishing the routine. Therefore, the control loop is closed using theimage stream. The edge debounce loop monitors the changing colors in theimage and sends a stable image signal when the pixels have stoppedchanging with respect to time

Fitness Filter 214: The fitness filter is an iterative evaluation thatuses the relative consistency of captures to evaluate quality ratherthan an absolute tolerance. This method is more flexible and accuratewhen trying to evaluate the wide range of colors and shapes inherent ina pharmaceutical setting. Fitness is evaluated by a computationallyintensive numerical method that measures the sharpness and clarity ofthe image. There are multiple, known strategies that may be employed forthis evaluation. The filter evaluates the image and returns value withina range that varies from a sharp image to a swatch of one color. Thisvalue is utilized as a metric to evaluate the fitness of the image.Multiple images are evaluated and the deviation of these images iscompared to a given tolerance. For example, the filter may determinethat all images are within a percentage of one another. If thepercentage is within the acceptable tolerance for a fit image it returnsa successful capture 216, otherwise the capture fails 218.

Employment Evaluation

Referring to FIG. 6, an image evaluation state diagram according to anexample embodiment is shown. In an example embodiment, the imageevaluation is implemented as a fast passive heuristic filter that usesthe previous training results as both an initial condition 220 as wellas a tolerance for the vial configuration in the order. It uses the sameintensive high-fidelity numerical method used in the training capturebut with one pass 222. After the results are compared to previouscaptures, it either returns a successful 224 or failed signal 226. Upona failed signal, it modifies the learning algorithm memory to take intoaccount the invalid capture.

The disclosed PV camera system reduces image capture time for vials onan automated dispensing line by employing a learning algorithm todetermine appropriate camera settings for various vial configurations.As vials are processed on the line, vial configuration data is used todetermine whether camera settings have been established for the vialconfigurations. When a known vial configuration is encountered, thelearned settings are retrieved from the library and applied to thecamera for imaging of the vial contents. When an unknown vialconfiguration is encountered, a training capture process is invoked tolearn the new configuration. When the previously unknown configurationis next encountered, the newly learned parameters may be applied.

While certain embodiments of the present invention are described indetail above, the scope of the invention is not to be considered limitedby such disclosure, and modifications are possible without departingfrom the spirit of the invention as evidenced by the following claims:

REFERENCES

-   1. Tong, Hang-Hang, Hongjiang Zhang, and Changshui Zhang. “Blur    Detection for Digital Images Using Wavelet Transform,” 2004 IEEE    International Conference on Multimedia and Expo, Vol. 1, pp. 17-20.

What is claimed is:
 1. A computerized method for capturing an image of adrug vial on an automated dispensing line comprising: (a) prior toreceiving an image of said drug vial from a camera, receiving at acomputer expected vial configuration data for said drug vial; (b)receiving at said computer at least one camera setting associated withsaid vial configuration data; (c) transmitting from said computer to acamera on said automated dispensing line said at least one camerasetting; (d) automatically configuring said camera according to saidsettings; (e) receiving at said computer from said camera an imagecaptured using said at least one camera setting; (f) prior to receivingan image of a second drug vial from a camera, receiving at said computervial configuration data for the second drug vial; (g) receiving at saidcomputer an indicator said vial configuration data for said second drugvial is not associated with at least one camera setting; (h) invoking atsaid computer a camera learning algorithm to identify at least onecamera setting associated with said vial configuration data for saidsecond drug vial; (i) receiving at said computer from said cameralearning algorithm at least one camera setting associated with said vialconfiguration data for said second drug vial; and (j) receiving at saidcomputer from said camera a second image captured using said at leastone camera setting associated with said vial configuration data for saidsecond drug vial.
 2. The computerized method of claim 1 wherein saidvial configuration data comprises: a drug identifier and a pill count.3. The computerized method of claim 1 wherein said at least one camerasetting includes focus and optionally one or more of: white red balance,white blue balance, gain, and exposure time.
 4. The computerized methodof claim 1 wherein said algorithm comprises causing the camera to: (1)focus a camera depth to a random level; (2) invoke an auto gain and anauto balancing feature; (3) wait for the image to achieve steady statewith respect to color; (4) initialize an auto focus feature; (5) waitfor the image to achieve steady state with respect to sharpness; and (6)capture a training image.
 5. The computerized system of claim 4 whereinreceiving at a computer vial configuration data for said drug vialcomprises: (1) receiving at said computer an identifier for said drugvial; (2) retrieving from a database using said identifier prescriptionorder data for said drug vial; and (3) identifying at said computer insaid prescription order data a drug identifier and a pill count.
 6. Thecomputerized system of claim 1, further comprising: (k) receiving atsaid computer said training image; and (l) evaluating at said computerimage quality of training image for conformance within specifiedtolerances.
 7. The computerized method of claim 6 further comprising:(m) storing at least one camera setting from said camera learningalgorithm with said vial configuration data for said second drug vial.8. The computerized method of claim 1 wherein receiving at a computervial configuration data for said drug vial comprises: (1) receiving atsaid computer an identifier for said drug vial; (2) retrieving from adatabase using said identifier prescription order data for said drugvial; and (3) identifying at said computer in said prescription orderdata a drug identifier and a pill count.
 9. A computerized system forcapturing an image of a drug vial on an automated dispensing linecomprising: (1) a line dispensing hardware controller that receives vialconfiguration data for said drug vial, said configuration data includinga drug identifier; (2) a camera host comprising a camera controllerthat: (a) receives said vial configuration data for said drug vial fromsaid line dispensing hardware controller; and (b) identifies at leastone camera setting associated with said vial configuration data, saidcamera setting including a focus setting; (3) a camera that: (a)receives from said camera host said at least one camera setting; (b)automatically configures itself according to said received setting(s);and (c) returns to said camera controller an image captured using saidat least one camera setting.
 10. The computerized system of claim 9wherein said vial configuration data comprises a drug identifier and apill count.
 11. The computerized system of claim 9 wherein said at leastone camera setting includes one or more of: white red balance, whiteblue balance, gain, and exposure time.
 12. The computerized system ofclaim 9 wherein the camera controller is adapted to: receive vialconfiguration data for a second drug vial prior to receiving an image ofa second drug vial from a camera; receive an indicator said vialconfiguration data for said second drug vial is not associated with atleast one camera setting; invoke a train image process of a cameralearning algorithm to identify at least one camera setting associatedwith said vial configuration data for said second drug vial, receivefrom said train image process at least one camera setting associatedwith said vial configuration data for said second drug vial; and receivefrom said camera a second image captured using said at least one camerasetting associated with said vial configuration data for said seconddrug vial.
 13. The computerized system of claim 12, wherein said trainimage process comprising causing the camera to: (i) focus a camera depthto a random level; (ii) invoke an auto gain and an auto balancingfeatures; (iii) wait for the image to achieve steady state with respectto color; (iv) initialize an auto focus feature; (v) wait for the imageto achieve steady state with respect to sharpness; (vi) capture atraining image.
 14. The computerized system of claim 13 wherein saidcamera controller further: (1) receives said training image; and (2)evaluates image quality of said training image for conformance withinspecified tolerances.
 15. The computerized system of claim 14 whereinsaid camera controller stores said at least one camera setting from saidtrain image process with said vial configuration data for said seconddrug vial.
 16. A computerized method for capturing an image of a drugvial on an automated dispensing line comprising: (a) storing in adatabase a plurality of vial configurations each consisting of a drugidentifier and a pill count and each of which is associated with atleast one camera setting; (b) receiving at a camera controller a vialconfiguration for said drug vial; (c) determining at said cameracontroller whether said vial configuration is in said database; (d) inresponse to determining said vial configuration is in said database: (1)locating said at least one camera setting associated with said vialsetting; (2) automatically configuring a camera using said camerasettings; and (3) capturing an image of said drug vial using said atleast one camera setting; and (e) in response to determining said vialconfiguration is not in said database, invoking a training process todetermine at least one new camera setting for said vial configuration,said training process comprises causing the camera to: (1) focus acamera depth to a random level; (2) initialize an auto focus feature;(3) capture a training image.
 17. The computerized method of claim 16wherein said at least one camera setting includes focus and optionallyone or more of: white red balance, white blue balance, gain, andexposure time.
 18. The computerized method of claim 16 furthercomprising storing in said database said at least one new camera settingfrom said training process with said vial configuration.